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Abstract 



Like institutions nationwide, DePaul currently is undergoing a transformation 
driven in part by the increasing diversification of its student body. As a result, this 
institution is challenged to meet higher student expectatioas of the educational 
experience. To meet these challenges, this institution annually conducts a student 
satisfaction survey assessing satisfaction with various facets of the university 
including student programs and services, environment and career preparation. This 
proposal will discuss the methodology for successfully conducting student 
satisfaction surveys, implications of these surveys for institutional research, and 
strategies for maximizing the use of student satisfaction data. Areas for continued 
improvement in conducting and utilizing student satisfaction surveys will also be 
discussed. 
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Student Satisfaction Surveys: Measurement and Utilization Issues 

To support the strategic planning process which strives to enhance quality and 
strengthen customer satisfaction, the Office of Institutional Planning and Research 
at DePaul University annually conducts a student satisfaction survey. Originally 
developed in 1990 by the Office of Student Affairs, the Office of Institutional 
Planning and Research has, since 1991, modified and administered the Student 
Satisfaction Survey in order to address three rising university initiatives. 

First, the survey was developed in anticipation of university adoption of a Total 
Quality Management (TQM) framework, a more customer-oriented management 
style. Although the TQM iriitiative has currently not been adopted, there is a strong 
movement toward quality assurance and business process re-engineering (PRE). 
Second, DePaul, like other institutions, is under pressure to adopt a more indicator- 
based planning and management process to meet both state and national initiatives. 
At the state level, an indicator-based planning process will address the Illinois 
Board of Higher Education's Priorities, Quality and Productivity (PQP) initiative 
which focuses on the clarification of institutional priorities in order to improve 
quality and productivity. At the national level, the development of student outcome 
indicators anticipates the North Central Accreditation process which will begin in 
1996. Third, attention to student needs is critical due to the rapid expansion of 
students and faculty at DePaul in recent years, not only at the structural level but in 
the services provided. This expansion has resulted in some dissatisfaction with the 
quality of student services. 

It is in this context that the Student Satisfaction Survey was developed. The 
survey assesses student satisfaction with various facets of the university and includes 
key sets of questions focusing on programs and services, university learning and 
social environment, university mission and values, educational preparation, transfer 
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intent, general satisfaction, attitudes toward coursework and student demograpliic 
information. The following sections discuss the conceptual framework, the selection 
of measures, the methodology for conducting student satisfaction surveys, the 
implications of these surveys for institutional research, and strategies for maximizing 
the use of student satisfaction data. Continued improvement in conducting and 
utilizing student satisfaction surveys will also be discussed. 
Conceptual Framework 

Like institutions nationwide, DePaul University currently is undergoing a 
transformation driven in part by the increased diversification of its student body. As 
a result, DePaul is challenged to meet higher student expectations of the 
educational experience. To meet these challenges and to anticipate initiatives at the 
University including TQM, indicator-based planning and student outcome 
assessment, ihe institution annually conducts a student satisfaction survey which 
supports institutional planning and quality development and assesses customer 
satisfaction in two ways {see figure 1). 

First, the survey assesses the service and learning environment , by examining 
satisfaction with broad academic areas and student services. The survey contains 
several content areas measuring student satisfaction with the service and learning 
environment including: 

* teaching and learning (such as academic quality, academic 
reputation and faculty quality) 

* academic and college support (such as academic advising and 
tutoring services) 

* class scheduling (such as catalogs and course schedules) 

* financial services (counselors and office assistance) 
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Service and Learning 
Environment: 

* teaching/learning 

* academic/college support 

* financial services 

* student services 
(library, computer lab, etc) 

* coursework 

* academic/personal experiences 

* educational prej^ration 



University 
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Institutional 
Affiliation: 

* catholicA^incentian 

presence 

* urban enrivonment 

* spirit of respect 

* personalism/caring 




Strategic Goals 
& Planning: 

* academic programs 

* commitment to diversity 

* academic support systems 

* personal environment 

* commitment to mission 
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* Student services (such as library, computer labs, etc) 

* coursework (in both general education and major) 

* academic and interpersonal experiences (including academic 
experiences and student and faculty interactions) 

* educational preparation (in career and graduate education). 

These content areas allow us to assess the quality of the student experience in a 
variety of academic and support services. This information ties directly to the 
strategic goal of enhancing the quality of DePaul's academic offerings. 

Second, these surveys assess students' institutional affiliation , by assessing the 
their support of and congruency with the institutional values and mission. During 
the rapid growth and diversification of the student body, it is important for DePaul 
to continue to effectively communicate its mission to students and to assess how 
effectively we are fulfilling our mission as an urban, Catholic comprehensive 
university. The content area that most directly focuses on the mission is that of 
University Enviromnent, measuring items related to: 

* catholic/Vincentian presence 

* urban environment 

* spirit of respect between different cultures 

* personalism and caring for students. 

Assessment of student perceptions of DePaul's mission feed directly into the 
strategic goal of providing an environment of pt xsonal growth and its conmiitment 
to a diverse student body. Survey coverage of both these areas, the service and 
learning environment and the institutional affiliation, provides the University with 
critical quality assessment information that feeds back into the planning process. 
Method for Conducting Student Satisfaction Surveys 

Overview. Since 1991, the Office of Institutional Planning and Research has 
annually conducted student satisfaction surveys during the spring term to assess 
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Students' attitudes of their educational experience. The survey samples 
undergraduates, graduates and law students in order to build a longitudinal 
database allowing us to examine problems and improvements in student satisfaction 
over time and to create a linked model to study causal relationships between 
important variables. 

Methodological Considerations. Special consideration is given to each step of 
the process in order to insure the success of the survey, including survey 
development, sample selection, survey preparation and mailing, and processing the 
data. 

Survey Development. The office involves the larger university community, 
especially the Provost, vice presidents and deans, in survey development. We have 
developed our own survey instead of using a commercially available survey because 
it provides the university with more institutional-specific information for quality 
improvement. The limitation of using a survey developed in-house, however, is that 
no national comparative data are available. 

In developing our survey, we strive for both consistency and flexibility. 
Consistenq/ is important in the questions to be included and the question wording in 
order to examine trends in student attitudes over time. Flexibility is also important 
in order to address topical concerns of the institution and to continue to develop a 
survey that provides the university with meaningful information. 

Sam ple Selection. In 1992, a sample of undergraduates, graduates and law 
students were selected to participate in the survey. In addition, all new 1992 
freshmen are surveyed in order to link the satisfaction data with other data collected 
by the university, including retention information and data from the freshman 
Cooperative Institutional Research Project (CIRP), and to prepare for a 
longitudinal senior study which is still being developed. 

Along with surveying all new freshmen to link freshman data with other 
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available data for a more integrated analysis of student experiences, the overall 
sample size is constantly being expanded in order to provide more detailed analysis 
of survey data, for example by majors, to meet the increasing demand for 
information for outcome assessment and unit-level discussions. 

In 1992, the overall response rate was 47.3%, with 56.5% for freshmen, 34.0% 
for continuing undergraduates, 56.5% graduates and 47,9% law students. 

Survey Preparation and Mailing . Several survey research procedures for 
preparing and mailing the .<;arvey are implemented to increase survey response rates. 
A high response rate is important not only in order to aciiieve a sample of students 
that is representative of the student body, but also to insure a database of student 
respondents that can be linked to data from other sources. For example, in order to 
study the freshman year experience based on survey information from the CIRP 
study and the Student Satisfaction Survey, it is necessary to receive Student 
Satisfaction Survey responses from a large percentage of CIRP respondents. In 
addition, in order to study trends in student graduation rates by linking the Studenc 
Satisfaction survey to a graduating senior survey, the greater the response rate for 
freshmen, the larger the possible sample is for the longitudinal freshman/senior 
sample. 

Students receive surveys in the mail during the spring term, giving special 
attention to the timing of the mailings in relation to the academic calendar so as not 
to mail surveys during finals. Follow-up mailings and other survey research 
procedures are used to maximize student response rates. 

Processing Survey Data. Once the data has been collected, we easily manage 
data entiy using SPSS/PC + Data Entry. Although the preparation is time- 
consuming, the end product is user-friendly for both the student worker who 
manages data entry and the analyst. 
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Implication for Institutional Research 

1. Student satisfaction surveys provide information that can be integrated into 
broader institutional agendas including the quest for quality and Total Quality 
Management (TQM). Student satisfaction surveys build on the central themes of 
total quality management (Chaffee & Sherr, 1992; Sherr & Lozier, 1991; Coate, 
1991), focusing on students as important constituents and identifying problem areas 
within the university. Using student satisfaction surveys, student programs and 
services that need quality improvement can be identified, action strategies to 
improve satisfaction can be developed and success can be measured via trends in 
multi-year student satisfaction data. 

For example, we develop a quality of services report analyzing student 
satisfaction with an extensive list of academic programs and student services. This 
report allowed us to identify service areas with which students are dissatisfied and 
develop strategies to improve satisfaction. In one instance, we found low responses 
to the Career Planning and Placement office indicating under-utilization. In 
addition, we found that although students felt DePaul prepared them for their 
chosen field, they were less certain about employment after graduation. 
Recommendations to increase student confidence in their futures focused on 
increasing awareness and use of these career counseling support programs and more 
effectively communicating how career counseling assists students in post-graduation 
employment. Improvements in awareness and use of Career Planning and 
Placement can be tracked with the survey. 

2. Institutional researchers can use year-to-year comparisons to track patterns 
in satisfaction over time. Although institutional researchers are often pressured to 
make numerical comparisons of changing satisfaction, researchers should focus on 
patterns of satisfaction over time because numerical comparisons may not be 
feasible due to inevitable changes in the instrument, sampling strategies or external 
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events that may effect responses. 

In one example, we experienced difficulty when we sought to compare reported 
1991 and 1992 student attitudes of programs and services because of changes in the 
sampling strategy. The 1991 survey included a sample of undergraduates, graduates, 
and law students, and the 1992 survey included all freshmen and samples of 
undergraduates, graduates, and law students. An additional analysis that weights 
the 1992 freshmen in the undergraduate sample similarly to the 1991 undergraduate 
sample is necessary in order to make numerical comparisons across year useful. 

In another example, the 1991-92 survey was mailed during the Chicago flood of 
April 1992. Because DePaul's downtown campus was severely affected by the flood, 
having to relocate students for about two weeks to classrooms at other downtown 
locations, student responses from this year were undoubtedly affected by this 
experience. Focusing on year-to-year trends in satisfaction will more acCwtrately 
capture student assessment, rather that focusing on numerical comparisons of the 
percentage agreement on each item. 

Because of the inevitable difficulties relating to changes in the instrument and 
the sampling strategy as well as ihe impact of uncontrollable external events, it is 
more practical to focus on changing patterns of satisfaction over time, 
acknowledging the limitations in numerical comparisons. Instead of comparing 
numerical changes in satisfaction on each specific service, we focus on clusters of 
items, general services, and the shifts in satisfaction of these clusters above or below 
a threshold of satisfaction. For example, instead of focusing on the increase from 
75% to 85% satisfaction in the item assessing quality of instruction, a specific 
service, we focus on the cluster of responses relating to academic quality, a more 
general service, and whether they are above a 75% threshold level, indicating that 
three in four students are satisfied. 

3. Institutional researchers can develop longitudinal databases by linking 
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Student satisfaction data to other research. Student satisfaction data will be linked 
to student entry data (for example, CIRP data) to probe the relationship between 
pre-coUege attitudes and actual college experiences. Student satisfaction data will 
also be linked to a graduating senior survey, still in development. 

Currently, we are linking satisfaction survey data with other institutional data, 
such as retention and academic performance data, to provide more integrated 
information about student retention. Preliminary results indicate notable 
differences in survey responses between 1991 freshmen who enrolled in Autumn 
1992 and those who did not enroll. Freshmen who did not re~enroll were less 
satisfied than those who did with their experiences with Admissions, Orientation, 
and the real-life preparation Df their educational experiences. 

The findings from linking the survey data with retention data suggest strategies 
for early identification of students at risk that cannot be gleaned from analysis of 
retention data alone. One strategy is to have students evaluate Admissions and 
Orientation early in the Autumn term and target retention efforts on those students 
who were dissatisfied. Another strategy is to more effectively communicate the 
relevance of the freshmen curriculum to student's future career and life experiences. 

4. Student satisfaction data provide institutional researchers with the 
opportunity to build a research framework that incorporates important variables 
that impact student satisfaction. We conducted a factor analysis using the items 
assessing University environment and correlated these factors with overall student 
satisfaction. From these items, we identified four central features of the student 
environment (caring for students, academic life, social and campus life, and mission 
and values) that drive satisfaction and used these to develop action strategies aimed 
at improving satisfaction. 

For example, for freshmen, the factor caring for students (which included the 
items such as personal attention and feeling welcome) was more strongly correlated 



Student Satisfaction 
12 



to overall satisfaction than the factors of academic life (with items such as academic 
atmosphere and overall instructional quality) or social and campus life (including 
good social atmosphere and places to relax), followed finally by the factor mission 
and values (including Catholic presence and conmiitment to urban mission). In 
contrast, for graduate students, academic life was most important to overall 
satisfaction, followed by mission and values and finally social and campus life . This 
analysis provides us with a framework for student satisfaction at our institution and 
suggests that strategies to strengthen student satisfaction should be tailored to 
different student groups. 

In addition, we included a multi-item scale measuring satisfaction with 
academic aspects (academic experiences and intellectual development) and social 
aspects (student-student and student-faculty interactions) of the educational 
experience (Pascarella and Terenzini, 1983). This scale allows us to study student 
experiences that have been identified as important in college student research 
(Pascarella & Terenzini, 1991) in order to further focus our strategies for improving 
satisfaction. 

Maximizing Student Satisfaction Data Utilization 

1. Institutional researchers can encourage utilization of student satisfaction 
data by providing specific, segmented information that can more easily be 
integrated in areas such as program review and outcomes assessment. Using 
student satisfaction surveys, institutional researchers can study a sample of students 
in order to pinpoint dissatisfied student groups and target specific problem areas. 
The institution is more likely to accept this specific information because it provides 
a focus for action. 

As described in several of the examples above, findings from the student 
satisfaction survey suggested several specific strategies for improvement. For 
instance, when survey data was linked to retention data, findings suggested specific 
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Strategies to enhance retention which include student evaluations of Admissions and 
Orientation to identify potentially at-risk students and include more effective 
communication of the relevance of the freshman curriculum. In addition, the factor 
analysis of survey data and the relationship of these factors to overall satisfaction for 
each student group provides the institution with priorities for improving satisfaction. 

2. Institutional researchers should collaborate with user offices to utilize focus 
groups or follow-up surveys to provide more in-depth analysis of identified problem 
areas* By design, student satisfaction surveys provide a global snapshot of student 
satisfaction and institutional researchers should use additional techniques to probe 
problem areas, thereby increasing the usefulness of the data. 

Much of the dissatisfaction in services is caused by a lack of collaboration in 
cross-functional areas. These focus groups and follow-up procedures help to 
pinpoint where the process breaks down and how certain university processes need 
to be re-engineered and streamlined. For example, the finding that only about one 
in three freshmen were satisfied with the timeliness of the grade reporting signaled 
an area where the process had broken down and quality needed to be improved. 
The Office of the Registrar conducted a focus group with students who indicated 
dissatisfaction with the timeliness of grade reporting in order to further study the 
problem and develop strategies for improvement. The focus group discussion, 
attended by representatives from several offices including the Registrar, Enrollment 
Management, Institutional Planning and Research and Information Systems, 
highlighted that both the problem of grade reporting and the solutions required 
involve several areas in the institution, including both administration and faculty. 
Improvements over time in student attitudes of grade reporting can be tracked with 
the survey. 

An office interested in conducting a follow-up analysis may desire to select a 
sample of students from the student body for the follow-up or may desire to further 
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Study the group of students who responded in a particular way to the survey. For 
example, because of the high number of students dissatisfied with the timeliness of 
grade reporting, the Office of the Registrar could have chosen to select a sample of 
students at large for a focus group. However, the Office chose to follow-up with 
those survey respondents who indicated that they were dissatisfied with grade 
reporting. 

In these cases where the original respondents are sought for follow-up analysis, 
institutional researchers must face the dilemma of confidentiality- Institutional 
researchers must develop procedures to balance the promise of confidentiality made 
to the respondents with the desire of user offices to go back to the original 
customer, in this case the responding student, to follow up on issues of service 
quality. 

Our office addresses the dilemma of confidentiality by requesting that user 
offices interested in following up on certain results go through our office to 
recontact students. We handle contact of the original respondents by explaining the 
circumstances surrounding the desire to further explore results and asking students 
if they would like to further participate. If they wish to participate, they are asked to 
contact our office and we pass their name to the office conducting the study; if they 
do not wish to participate they need not call and we do not contact them further. 

3. Audience-specific communication strategies can be developed to assist the 
University in using student satisfaction data and better understanding the utility of 
student satisfaction surveys. We focus on several types of strategies to enhance 
audience-specific communication: 1) reporting strategies, 2) graphical presentations 
of data, 3) utilizing current technology to develop user tools, and 4) development of 
a student database focused on strategic issues. 

First, strategies for reporting data should focus on presenting information to 
targeted university audiences that provides an integrated analysis of the issues. 
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These reports should include a quality of service report as well as several topical 
reports focusing on specific issues from the data and the university strategic 
planning agenda. These issues-oriented reports, which may include topics such as 
student diversity and geographic origin of students, should integrate information 
from other sources to provide an in-depth, integrated analysis of the issues, relying 
on graphics to enhance data presentation, and feed information back into the 
strategic planning process. 

Two difficulties exist in reporting student satisfaction data. First, the reports 
need to provide a focused analysis of the specific issues relating to strategic 
planning. However, because of the richness of the integrated datasets, it may be 
difficult *o focus the data around specific issues. Second, it may be difficult to 
provide user offices with evidence of improvement that is both statistical and 
practical. 

Second, by using graphical presentation of data, we can focus on broader issues, 
such as trends in diversity at the university level, and can effectively summarize data 
relating to these broad issues from a variety of data sources including the Student 
Satisfaction Survey for members in the university community who may not read or 
have access to the available reports. We use graphics in our reports to highlight 
trends in the data, and we provide presentations of the data based on these 
graphical analyses to audiences such as the Board of Trustees. 

These audiences, however, are interested not only in University data (including 
customer satisfaction data), but comparative data, for example national student 
information. Because we develop our own student satisfaction surveys, direct 
comparative data is often unavailable. 

The third strategy is the utilization of available technology to develop tools that 
allow offices further access to the data for more specialized analyses of specific data 
questions. We plan to provide student satisfaction data and reports in a network 
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environment to make access easier. This would allow offices to integrate survey 
data into their work by easily tailoring survey data tables for their own reports and 
presentations. 

We also have developed a Quattro Pro data base tool that allows individuals to 
easily query the database and generate specific crosstabulations. This allows offices 
such as Student Affairs to probe more specifically questions they have about specific 
populations of students. 

Increasing access to data for user offices with specific data questions may be 
problematic, however, if offices overestimate the depth of student satisfaction data. 
This can occur, for instance, if an office overemphasizes the low satisfaction of a 
service area that has very low response rates. 

Finally, we are developing 2i database organized around strategic university issues, 
including such sources as student satisfaction data, retention data and academic 
indicator data. This will help university planners focus on important university 
issues, for example student retention, and allow the university planning committees 
such as the Retention Committee access to critical student data. 

In summary, by developing audience-specific communication strategies, 
institutional researchers not only provide oral and written reports of the data to the 
institution, but utilize available technology to provide offices hands-on access to 
information that can feed directly into specific projects. Through increased 
awareness and use of the Student Satisfaction Survey data, offices can develop a 
better understanding the strengths and limitations of the Student Satisfaction 
Survey. 

Areas for Continued Improvement 

In summary, this paper discusses the methodology of conducting student 
satisfaction surveys, the implications of these surveys for institutional research, and 
strategies for maximizing their use. From this discussion, the following section 
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highlights a few general areas of perennial concern for institutional researchers 
conducting student satisfaction surveys. 

1. Involve users of the survey in the survey development. This builds 
ownership in survey results and helps to insure that the survey will provide useful 
information for the university community that can feed into the strategic plarming 
process. 

2. Develop a methodology for conducting the survey that is both consistent and 
flexible, linking the survey measures to the university strategic goals and objectives. 

This will help insure the ability to draw longitudinal conclusions as well as address 
important topical concerns in order to provide valuable information for university 
planning. 

3. Maximize response rate by using a variety of survey response techniques. 

These include such things as length, visual presentation of the survey and follow-up 
mailings. This is especially important when data linking and longitudinal analysis is 
desired because of respondent attrition. 

4. Make data easy to understand. Develop an audience-specific approach to 
reports and presentations, using graphics to enhance understanding of results. 
Integrate survey data with data from other sources to provide comprehensive 
analyses of important university issues. 

5. Make data useful by linking results to policy issues. This starts from 
developing a survey that asks the right questions. Once the data is collected, 
usefulness can be enhanced by using technology to create easy access to data, 
providing practical suggestions, and integrating data from multiple sources to 
develop a broader understanding of the issues. This data, focusing on important 
strategic planning issues relating to the assessment of the academic and student 
services as well as institutional affiliation, will be useful in assessing the quality of 
the educational experience as well as in the institution's planning process. 
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